Visual body pose analysis for human-computer interaction

Human-Computer Interaction (HCI) is the study of interaction between people (users) and computers. The recent advances in computing technology push the interest in human-computer interaction in other ways than the traditional keyboard, mouse or keypad devices. The work presented in this thesis uses computer vision to enhance the HCI, by introducing novel real-time and marker-less gesture and body movement-based systems. Real-time systems have a high refresh rate and minimal latency, providing the user with smooth and instantaneous interaction with the system. Marker-less systems allow a natural interaction without wearing special markers or special tracking suits, which are generally required in modern day tracking systems. The systems described in this thesis aim to achieve this real-time marker-less HCI. They are based on vision and built with standard computers equipped with standard color cameras. The goal set for this work is hand gesture-based interaction with large displays, as well as full body pose recognition for interaction where the user is immersed in a virtual environment. The systems described in this thesis can be divided into three components: (1) preparing the input for detection and recognition, which includes segmentation and reconstruction; (2) detecting of body parts and recognition of body poses and hand gestures; (3) using the detection/recognition to steer the application. These three components are reflected in chapters 2 to 4 of this thesis. Segmentation and Reconstruction. The first part of the thesis provides a brief summary of foreground-background segmentation, skin color segmentation and 3D hull reconstruction. Skin color 4 segmentation usually suffers from changes in lighting and of the user. Therefore, a novel and improved skin color segmentation algorithm is introduced, which combines an offline and an online model. The online skin color model is updated at run-time based on color information taken from the face region of the user. Detection and Recognition. The second part of the thesis starts with a summary of how the face, eye, hand and finger locations can be detected in a camera image. Then, a novel body pose recognition system is introduced based on Linear Discriminant Analysis (LDA) and Average Neighborhood Margin Maximiza-tion (ANMM). This system is able to classify poses based either on 2D silhouettes or 3D hulls. Using a similar technique, a novel hand gesture recognition system is introduced. Both the body pose and hand gesture recognition systems are improved for speed with the help of Haarlets. A novel Haarlet training algorithm is …